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DETAILED ACTION 



1 . This communication is responsive to Amendment A, filed 9/25/2002. 

2. Claims 1-50 are pending in this application. Claims 1, 3, 12, 16, 23, 28, 32, 42, and 47 
are independent claims. In the Amendment A, claims 14 and 19 were amended, and claims 49- 
50 were added. This action is made Final. 

3. The text of those sections of Title 35, U.S. Code not included in this action can be found 
in a prior Office action. 



4. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 

basis for the rejections under this section made in this Office action: 
A person shall be entitled to a patent unless - 

(e) the invention was described in a patent granted on an application for patent by another filed in the United 
States before the invention thereof by the applicant for patent, or on an international application by another who 
has fulfilled the requirements of paragraphs (1), (2), and (4) of section 371(c) of this title before the invention 
thereof by the applicant for patent. 

The changes made to 35 U.S.C. 102(e) by the American Inventors Protection Act of 1999 
(AIPA) and the Intellectual Property and High Technology Technical Amendments Act of 2002 
do not apply when the reference is a U.S. patent resulting directly or indirectly from an 
international application filed before November 29, 2000. Therefore, the prior art date of the 
reference is determined under 35 U.S.C. 102(e) prior to the amendment by the AIPA (pre-AIPA 
35 U.S.C. 102(e)). 



Claim Rejections - 35 USC§102 
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5. Claims 1, 3, 10-12, 14, 23, 27-33, 36, 39-42, and 44-46 are rejected under 35 
U.S.C. 102(e) as being anticipated by Eberman et al. ("Eberman", US 6,173,287 Bl), 

As per independent claim 1, Eberman teaches a networked client/server system, 
comprising: 

a network annotation server (fig. lA; col. 2, lines 35-36; indexed database server 32); 

a network media server (fig. 1 A; media database server / library 28)\ 

a client that communicates with both the annotation server and the media server over a 
data communications network (fig. 1 A; browser client 20)\ 

multimedia content available from the media server (col. 2, lines 36-37); 

a plurality of annotations, corresponding to the multimedia content, available from the 
annotation server, each of the plurality of annotations including annotation content and a 
temporal range identifier that identifies a segment of the multimedia content to which the 
annotation corresponds (fig. 10; col. 17, lines 1-48); and 

wherein the client supports a graphical user interface that presents a plurality of 
annotation identifiers corresponding to the multimedia content and that enables a user to request 
selected ones of the plurality of annotations, based on the plurality of annotation identifiers, and 
to render the requested annotations (col. 8, lines 18-32). 

As per independent claims 3 and 10-11, Eberman teaches a graphical user interface for 
adding annotations to an annotation database from a network client, the graphical user interface 
comprising: 

an annotation content field via which a user can enter content for a new annotation to the 
network client; and an annotation type selector presenting a plurality of annotation media types, 
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whereupon selection of one of the plurality of annotation types causes the network client to 
change presentation of the annotation content field based on the selected annotation type; an add 
button, whereupon selection of the add button causes the network client to forward the content 
from the annotation content field to an annotation server to be added to an annotation database; 
and temporal range information identifying a segment of media content to which the annotation 
corresponds (figs. 8-10; col. 16, lines 11 - 67). 

Eberman does not expressly disclose an annotation content field, an annotation type 
selector, an add button and temporal range information in a GUI as claimed. However, these 
GUI elements would have been inherently part of Eberman' s GUI so as to provide users with a 
means for entering the annotation information as depicted in the database tables as shown in 
figures 8-10. 

As per claim 12, Eberman teaches a graphical user interface at a network cUent to search 
for annotations corresponding to media content in an annotation database, the graphical user 
interface comprising: 

an annotation set selector via which a user can identify one or more of a plurality of 
annotation sets to be searched; a search criteria portion via which a user can identify search 
criteria; and a query button, wherein actuation of the query button causes the network client to 
forward the identified one or more annotation sets and the identified search criteria to an 
annotation server to search the annotation database (col. 8, lines 18-32). 

Claims 23 and 27 are similar in scope to claim 10, and are therefore rejected under 
similar rationale. 

Claim 28 is similar in scope to claim 27, and is therefore rejected under similar rationale. 
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As per claim 29, Eberman teaches the data associated the user interface comprises an 
annotation set identifier (col. 7, Unes 46-51; object identification number). 

Claim 30 is similar in scope to claim 1 1, and is therefore rejected under similar rationale. 

As per claim 31, Eberman teaches one or more computer-readable memories containing a 
computer program that is executable by a processor to perform the method recited in claim 28 
(abstract). 

Claims 32, 36 and 39-40 are similar in scope to claim 30, and is therefore rejected under 
similar rationale. 

Claim 33 is similar in scope to claim 30, and is therefore rejected under similar rationale. 

Claim 41 is similar in scope to claim 31, and is therefore rejected under similar rationale. 

Claim 42 is similar in scope to claim 32, and would have been therefore rejected under 
similar rationale. Eberman further teaches the presenting of a plurality of likely temporal 
locations of the media content to associate the new annotation with (figs. 12-13; col. 15, lines 4- 

11). 

As per claims 44-45, Eberman teaches for each of the plurality of likely temporal 
locations, displaying a visual indication of the likely temporal location, wherein the visual 
indication includes one or more of: a video frame of the media content, a numerical presentation 
time of the media content, and an indicator on a graphical time bar (figs 12-13). 

Claim 46 is similar in scope to claim 31, and is therefore rejected under similar rationale. 





Application/Control Number: 09/396,706 
Art Unit: 2174 



Page 6 



Claim Rejections - 35 (JSC §103 



6. Claims 4-6, 8, 14-22, 24, 26, 34-35, 37-38, and 49-50 are rejected under 35 U.S.C. 103(a) 
as being unpatentable over Eberman et al. ("Eberman", US 6,173,287 Bl). 

As per claims 4-6, by disclosing that "annotations could be, for example, a piece of 
information relating to data in a particular form such as, for example audio or video data" (col. 2, 
lines 29-31), it would have been obvious to an artisan that Eberman teaches the plurality of 
annotation types to include audio, video, and URL, as well as suggests text to be an option, 
wherein the annotation content field comprises a uniform resource locator (URL)/text field when 
an URL/text type is selected respectively. Text as an option would provide an important form 
for all annotations of multimedia contents. 

As per claim 8, although Eberman teaches a plurality of annotation type selector, 
Eberman does not explicitly disclose the type selector to comprise a radio button for each of the 
plurality of annotation types. Official Notice is given that the use of radio button as a means for 
selecting an item from a list of items in a menu is well known in the art. It would have been 
obvious to an artisan at the time of the invention to combine the use of radio buttons with 
Eberman GUI in order to provide users with an effective means for selecting an item that is 
mutually exclusive from a list of items in a menu. 

As per claim 14, Eberman does not disclose the search criteria portion to include an 
annotation creation date entry field. However, Official Notice is given that associating creation 
date/time information to annotation fields as well as the use thereof as search criteria are well 
known in the art. Furthermore, by disclosing an "advanced search" option (col 21, lines 8-9), 
Eberman also suggests for advanced search features. It would have been obvious to an artisan at 
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the time of the invention to include this information with Eberman's method in order to provide 
additional useful information regarding when the annotations are created as well as for searching 
annotations that were created during a specific timeframe. 

As per claim 15, Eberman teaches the GUI to comprise a detail level selector via which a 
user can indicate an amount of data to be displayed for each annotation that matches the search 
criteria (col. 22, lines 63-67). 

As per claims 16-18, Eberman teaches a graphical user interface for viewing, at a 
network client, annotations corresponding to media content, the graphical user interface 
comprising: an annotation identifier list via which an identifier for each of a plurality of 
annotations corresponding to the media content is displayed, the identifier including an 
indication of a type of content included in the annotation; an actuation mechanism to enable a 
user to select one of the annotation identifiers; wherein the type of content includes one or more 
of audio content, text content, video content, and uniform resource locator (URL) content; and 
wherein the identifier for an annotation includes one or more of an indication of an author of the 
annotation, an indication of an annotation set that the annotation belongs to, an indication of a 
date the annotation was created, and a summary of the annotation (fig. 12; col. 22, lines 23-46). 

Eberman does not explicitly disclose the selection of one of the annotation identifiers to 
cause the network client to highlight the annotation identifier. Official Notice is given that 
highlighting an item as a result of the item being selected is well known in the art. It would have 
been obvious to an artisan to include this feature with Eberman's GUI in order to provide users 
with a means for distinguishing a selected identifier from the unselected identifiers. 



Application/Control Number: 09/396,706 Page 8 

Art Unit: 2174 

As per claim 19, Eberman teaches a mechanism to identify a particular identifier that 
corresponds to an annotation of the plurality of annotations with a temporal range having a 
beginning time closest to the current presentation time of the media content (col. 8, line 33 - col. 
9, line 31; and col. 17, line 49 - col. 18, Hne 16; the subjected identifier would be the very first 
identifier shown on the results query page). 

As per claim 20, Eberman does not disclose an arrow being used to indicate the particular 
identifier. Official Notice is given that the use of a visual icon associated with an item in a GUI 
is well known in the art. It would have been obvious to an artisan at the time of the invention to 
combine the use an icon such as that of an arrow with Eberman' s GUI to further provide users to 
quickly identify a particular identifier through such a visually means. 

As per claim 21, Eberman teaches the GUI to further comprise a preview portion via 
which annotation content for a selected one of the plurality of annotations is displayed (col 22, 
lines 47-50). 

As per claim 22, Eberman does not disclose a menu including a plurality of options 
identifying criteria to be used to order the annotation identifiers in the annotation identifier 
portion, whereupon selection of one of the plurality of options by a user causes the network 
client to arrange the annotation identifiers in the annotation identifier list in accordance with the 
criteria of the selected option. Official Notice is given that sorting annotation identifiers 
according to a preference options is well known in the art. It would have been obvious to an 
artisan at the time of the invention to include this feature with Eberman' s GUI in order to 
provide users with different sorting choices. 
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Claims 24 and 26 are similar in scope to claims 19 and 21 respectively, and are therefore 
rejected under similar rationale. 

As per claims 34-35, Eberman does not specifically disclose the use of a means to 
indicate acceptance or rejection while in the process of associating the new annotation with a 
temporal location of the media content. Official Notice is given that the use of a confirmation 
means, such as <0K> and <CANCEL>, at the end of a process is well known in the art. It 
would have been obvious to an artisan at the time of the invention to include such a means with 
Eberman' s method in order to allow users with a chance to reconsider before committing the 
new annotation with a temporal location of a portion of the media content, or to associate the 
new annotation with another different portion of the media content. 

As per claims 37-38, Eberman does not expUcitly disclose the presenting to comprise 
displaying a video frame or a presentation time of the media content corresponding to the likely 
temporal location. Official Notice is given that the displaying of video frames and presentation 
time corresponding to a media location being reviewed for annotation is well known in the art. It 
would have been obvious to an artisan at the time of the invention to include such displays with 
Eberman' s method in order to provide users with a visual means for verifying pertinent 
information about the media content being considered for the new annotations. 

As per claim 49, although Eberman does not explicitly disclose each annotation identifier 
of the plurality of annotation identifiers to comprise an annotation subject line of an annotation 
corresponding to the multimedia content, it would have been obvious to an artisan at the time of 
the invention that some sort of subject line must be created-as/associated-with annotation 
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identifiers in order to describe what the annotation is about. Moreover, such a description could 
also be effectively used as part of a search criteria. 

Claim 50 is similar in scope to claim 49, and is therefore rejected under similar rationale. 

7. Claims 2 and 25 are rejected under 35 U.S.C. 103(a) as being unpatentable over Eberman 
et al. ("Eberman", US 6,173,287 Bl) in view of Sutton et al. ("Sutton", US 4,649,499). 

As per claim 2, although Eberman teaches the graphical user interface to include a 
plurality of option buttons, whereupon selection of one of the plurality of the option buttons, an 
associated action corresponding to the selected button is taken (fig. 11; buttons 174-178), 
Eberman does not teach the buttons to be user-configurable. Sutton teaches a GUI, wherein a 
series of user-configurable buttons associated with respective user-defined actions are provided 
(col. 3, hnes 44-46). It would have been obvious to an artisan at the time of the invention to 
combine Sutton's teaching with Eberman' s system in order to further provide more flexibility to 
user's capability in defining selectable options according to preference. 

Claim 25 is similar in scope to claim 2, and is therefore rejected under similar rationale. 

8. Claims 7 and 43 are rejected under 35 U.S.C. 103(a) as being unpatentable over Eberman 
et al. ("Eberman", US 6,173,287 Bl) in view of King et al. ("King", US 5,600,775). 

As per claim 7, Eberman does not disclose the annotation content field to comprise, when 
an audio type is selected, a plurality of audio controls including one or more of record, stop, 
pause, play, fast forward, and rewind. King teaches a method for annotating multimedia 
comprising a GUI comprising graphical elements such as buttons which provide control 
functions for media annotation such as record, stop, pause, play, fast forward, and rewind (fig. 
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2). It would have been obvious to an artisan at the time to combine King's control functions 
v^ith Eberman's GUI in order to provide a means for controlling annotating functions. 

As per claim 43, although Eberman teaches the presenting the plurality of likely temporal 
locations to comprise identifying a different one of the plurality of likely temporal locations to 
the user, Eberman does not teach the use of a rewind button to be actuated each time a different 
temporal location is to be identified. King teaches a rewind button for media annotation (fig. 2; 
button 53). It would have been obvious to an artisan at the time to combine King's rewind button 
with Eberman's method in order to provide a means for changing to a different temporal location 
as a likely candidate to be associated with the new annotation. 

9. Claim 9 is rejected under 35 U.S.C. 103(a) as being unpatentable over Eberman et al. 
("Eberman", US 6,173,287 Bl) in view of Hou et al. ("Hou", US 5,838,313). 

As per claim 9, although Eberman teaches the use of an email feature (col. 9, lines 50- 
61), Eberman does not explicitly disclose an email field to identify a recipient to receive an email 
notification of the new annotation. Hou teaches a multimedia system wherein a dynamic 
annotation handler allows a user to record/playback annotations from the user input as well as 
forwarding the annotations to other individuals via emails (col. 2, line 56 - col. 3, line 13). It 
would have been obvious to an artisan at the time to combine Hou's emailing feature with 
Eberman's GUI in order to provide a means for forwarding/informing pertinent recipients of the 
new annotations. 
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Response to Arguments 



10. Applicant's arguments with respect to the amended/new claims 14 and 49-50 have been 
considered but are moot in view of the new ground(s) of rejection. 

1 1 . Applicant's arguments with respect to all other claims have been fully considered but they 
are not persuasive. 

Applicant argues that there is no disclosure/suggestion/discussion in Eberman: (a) for a 
GUI to render requested annotations as indicated in claim 1 and other similarly claimed claims, 
since Eberman' s discloses the rendering of digital representation using annotations as a basis for 



searching for objects; (b) of a graphical user interface for adding annotations, and how a human 
annotator generates/adds annotations as indicated in claim 3 and other similarly claimed claims; 
(c) of an annotation set selector via which a user can identify one or more of a plurality of 
annotation sets to be searched as indicated in claim 12 and other similarly claimed claims; (d) for 
the reviews and updates of annotations to be new annotations, but rather for previously generated 
annotation as indicated in claim 32 and other similarly claimed claims; (e) of a GUI comprising 
an annotation identifier list that includes one or more of the identifier for an annotation including 
an author of the annotation, an indication of an annotation set that the annotation belongs to, an 
indication of a date the annotation was created, and a summary of the annotation as indicated in 
claim 18 and other similarly claimed claims. Applicant fiirther argues that: (f) all secondary 
references cited in claims 2, 25, 7, 43, 9, and 43 (Sutton King et al. and Hou) do not cure the 
deficiencies of Eberman as claimed in the independent claims. 
The examiner disagrees for the following reasons: 



Application/Control Number: 09/396,706 Page 13 

Art Unit: 2174 

Per (a), it is noted that the claim language only requires for a GUI to present "a plurality 
of annotation identifiers... and that enables a user to request selected ones of the plurality of 
annotations, based on the plurality of annotation identifiers, and to render the requested 
annotations." Eberman's GUI as shown on fig. 12 still reads on the limitations in that the GUI: 
provides users with annotation identifiers (such as "history"); users are able to select an 
annotation fi*om the five annotations listed; and the annotations are rendered after users select 
one of the five annotation choices. 

Per (b), by disclosing that "Annotations may be generated. . .by trusted human annotators" 
as well as previously generated annotations may be reviewed and updated by human annotators 
(col. 15, lines 5-11) in accordance to the data structure as defined on figures 8-10, an user 
interface comprising various fields and/or controls would have been inherent in Eberman's 
system in order to allow users a an interface means for creating (generating/populating) 
annotations and their associated information, such as annotation contents and types, as defined in 
the tables of figs. 8-10. 

Per (c), in contrast to the description in the specification which describes the GUI to 
display a list of annotation sets (specification, page 17), the claim language merely requires for 
an annotation set selector via which a user can identify one or more of a plurality of annotation 
sets to be searched. Eberman meets the claim limitation since Eberman provides a means for an 
user to request/query the system to select annotation sets fi-om the database matching the search 
criteria (fig. 12; annotation sets of ''commission*' and ''history"). 

Per (d), argument is moot in view of the response to the argument in (b). 
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Per (e), Eberman discloses that the GUI as shown in fig. 12 comprises an indication of an 
annotation set that the annotation belongs to (e.g. ''history''). 

Per (f), in response to applicant's arguments against the references individually, one 
cannot show nonobviousness by attacking references individually where the rejections are based 
on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In 
re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). 



Allowable Subject Matter 

12. Claim 13 is objected to as being dependent upon a rejected base claim, but would be 
allowable if rewritten in independent form including all of the limitations of the base claim and 
any intervening claims. 

13. Claims 47-48 are allowed 

14. The following is an examiner's statement of reasons for allowance: 

The prior art made of record fails to anticipate or make obvious the claimed invention. 
Specifically, the prior art fails to teach, in combination with the remaining elements: 

the graphical user interface to comprise a target check box, whereupon selection of the 
target check box causes the network client to add, as an additional search criteria, a media 
content identifier as recited in claim 13; and the computer program to perform the fiinction of 
selecting, based on the user request time, a presentation time of the media content to associate 
the new annotation with, wherein the presentation time is a different time than the user request 
time as recited in claim 15. 
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While Eberman teaches a multimedia annotation method wherein users are inherently 
provided with an user interface to generate/add, review and update annotations, Eberman fails to 
teach the step as cited above. 

Any comments considered necessary by applicant must be submitted no later than the 
payment of the issue fee and, to avoid processing delays, should preferably accompany the issue 
fee. Such submissions should be clearly labeled "Comments on Statement of Reasons for 
Allowance." 



Conclusion 

15. TfflS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1. 136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the mailing 
date of this final action. 
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Inquires 



16. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Sy Luu whose telephone number is (703) 305-0409. The 
examiner can normally be reached on Monday - Thursday from 6:30 am to 4:00 pm (EST). The 
examiner can also be reached on alternate Friday. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Kristine Kincaid, can be reached on (703) 308-0640. 

The fax number for the organization where this application or proceeding is assigned are 
as follows: 

(703) 746-7238 [After Final Communication] 

(703) 746-7239 [Official Communication] 

(703) 746-7240 [For status inquiries, Draft Communication] 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the receptionist whose telephone number is (703) 305-3900. 




SyD. I5mr 
PRIMARY EXAMINER 
December 12, 2002 



